You should be using uncoated inserts or end mills for aluminium. Ideally look for the "honed", "lapped" or "polished" cutters with razor sharp edges (but be careful handling them!). Something like this https://www.cutwel.co.uk/milling-too...l-apfe3-korloy For inserts, the "G" ones with "higher tolerance" are sharp and polished eg CCGT rather CCMT, usually uncoated.

You may find that taking a deeper cut (higher feed rare) actually gives a better finish than a "spring pass". I use coolant or WD40 for best finishes. It's also good to ensure the chips are cleared so they don't get rubbed against the workpiece, using air blast.